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ABSTRACT 


This report considers the stochastic, inf inite- time , discrete output 
feedback problem for time-invariant linear systems. Two sets of sufficient 
conditions for the existence of a stable, globally optimal solution are 
presented. An expression for the total change in the cost function due to 
a change in the feedback gain is obtained. This expression is used to show 
that a sequence of gains can be obtained by an algorithm, so that the 
corresponding cost sequence is monotonically decreasing and the corresponding 
sequence of the cost gradients converges to zero. The algorithm is guaranteed 
to obtain a critical point of the cost function. The computational steps 
necessary to implement the algorithm on a computer are presented. The results 
are applied to a digital outer-loop flight control problem. The numerical 
results for this 13th order problem indicate a rate of convergence consid- 
erably faster than two other algorithms used for comparison when they 
converge. 
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I, INTRODUCTION 


Various formulations of the optimal output feedback problem have received 
considerable attention over the last two decades [1] - [21]. Loosely, the 
problem consists of finding a control law optimal with respect to a (usually) 
quadratic cost function, for a given linear system when the control is con- 
strained to be linear in the instantaneous ^outputs” i.e., a specified set 
of measurements or states. Various forms of the problem correspond to 
whether the system is continuous or discrete, stochastic or deterministic, 
whether the optimization interval is finite or infinite, and whether the 
control law and system are time- invariant or time-varying. More subtle vari- 
ations of the problem, such as the specific treatment of white measurement 
noise in continuous problems (e.g., compare [11] and [12]), have also been 
formulated. 

The optimal output feedback problem is a significant extension of the 
optimal quadratic **full state feedback” problem, [22]. The optimal output 
feedback formulation addresses some of the limitations encountered in prac- 
tical systems and provides a flexibility useful in configuring the control 
law for ease of implementation. In many cases involving complex, high 
order systems, all the states are not measured. Optimal output feedback 
provides a method of designing simple control laws for such situations. More 
importantly, output feedback provides a method of designing control laws in 
cases where it is desirable not to feed back some states, even if measurements 
are available. For example, in a complex system such as an aircraft, the 
aircraft’s aerodynamics are coupled with subsystems, such as engine dynamics 
and hydraulic systems, which drive the control actuators. It is often desir- 
able not to feed back all the subsystem states, (e.g., the hydraulic fuel 
flow rate) in order to track a specified flight path. 



Optimal output feedback provides a modern control alternative to the 
classical frequency-domain methods of designing simple, low order dynamic 
compensators and outer-loop compensators (i.e., major/minor loop compensation 
[23]). It is well known that the fixed order dynamic compensator problem can 
be imbedded in the output feedback problem [6], [8]. A major design flexi- 
bility is therefore gained using optimal output feedback when compared to 
the LQG approach which results in a full order Kalman filter as a dynamic 
compensator. 

The stochastic output feedback problem provides a systematic approach for 
increasing or decreasing specific sets of feedback gains by appropriately 
varying the measurement and plant noise covariances. This flexibility is due 
to the fact that the separation of estimation and control present in the LQG 
solution does not hold for the optimal stochastic output feedback problem. 

Thus, if a measurement is noisy, the fact that part of this noise will be 
introduced into the system through the control is recognized, and the corre- 
sponding gains are automatically reduced. Output feedback has various other 
advantages which are often useful in pactical control designs. 

The necessary conditions for the various formulations of the optimal 
output feedback problem are well-known, [1] - [13]. The resulting equations 
are coupled non-linear matrix equations. Various algorithms to solve these 
equations have been suggested. These algorithms include sequentially solving 
the non-linear matrix equations as in [4] , sequentially solving a set of 
linear matrix equations as in [5] and [13], gradient based searches to reach 
the cost function minimum as in [14], [17] - [19], and non-gradient based 
search procedures as in [15] and [21]. It is recognized that the sequential 
algorithms are **fast” when compared to the search algorithms, [21], but their 
convergence is, at best, not guaranteed [20]. The search algorithms can con- 
verge to a critical point, [19], [17], but require large amounts of computation 
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time that increase significantly as the order of the plant increases. The 
unavailability of a fast, convergent and numerically reliable algorithm has, 
in the authors* opinion, been a major hindrance to the successful application 
of the optimal output feedback design approach. 

This report considers the stochastic, inf ini te- time, discrete, output 
feedback problem for time- invariant linear systems. The problem is formulated 
in Section II. Sufficient conditions for the existence of a stable globally 
optimal solution to the infinite time optimization problem is presented in 
Section III. Section IV presents necessary conditions for an output feedback 
gain to minimize the infinite time quadratic optimization cost function. Also 
shown in Section IV is an exact expression for the change in the value of the 
cost function due to a change in the output feedback gain. This exact expres- 
sion is used in Section V to show that a sequence of gains, defined by an 
algorithm, monotonically decreases the infinite time quadratic cost function 
while correspondingly causing the gradient of the cost function to approach 
zero. The algorithm can be started from any stabilizing gain (i.e., any gain 
which stabilizes the closed-loop system) and is guaranteed to obtain a critical 
point. However, as with the other algorithms mentioned, this critical point 
need not be a global minimum of the cost function. The steps in the algorithm 
are given at the end of Section V. An outer-loop digital control problem is 
used in Section VI to compare the algorithm to other methods for obtaining the 
optimal output feedback gain. Output feedback gain variations with different 
choices for the measurement noise and process noise covariance are also shown 
in the example in Section VI. The numerical results for this 13th order system 
confirm the convergence properties and indicate a rate of convergence consid- 
erably faster than the other algorithms tested when the latter converge. 
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II. PROBLEM FORMULATION 


Consider the discrete, t ime- invar iant , stochastic system described by 

=‘k+i ■ '*' ''k + "k- 

where x^, y^^, represent the state, measurement (output), control vectors, 
respectively, and w , v, are the plant, measurement noise vectors, respectively, 

K. K 

satisfying the conditions 




E(wp = 

0, 

E ( w- w ! ) 
k 3' 
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kj’ 

(3) 

E(vp = 

0. 
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II 
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= s . 

o 

(5) 


The class of control laws considered is restricted to be of the form 


"k = - ^ 


i.e., the feedback of measurements or selected state components through a 
constant gain. It should be noted that it is not necessary that the noise 

/N /\ 

covariance, W and V, be positive definite or that the mean of the initial 
condition, x^, be zero, although some of these conditions may be used to show 
specific properties of the optimal solution. Now consider a cost function 
of the form 


'=<iio "1+1 "i+i + “1 K up. N > 0 


For the case of a deterministic system (i.e., W = 0, V = 0), as N 


The prime. 


', denotes the transpose of a vector or matrix. 
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J^(K) remains finite when the control law, (6), stabilizes the system* However, 
in the stochastic case, g^ows without bounds (except for some trivial 

cases) as can be seen by the inequality 

Q R u^) ^ tr I Q W } + tr I K' R K V } ^ 0, i ^ 0. (8) 


Thus, to treat the infinite optimization interval for the stochastic 
case, it is necessary to modify the cost function. A natural selection is to 
consider the average cost 


- 2(SIT Jo " 1+1 '* "i+i + “1 K ^ 


J(K) = lim J-,(K), K < V, 
N-k» M 


(9) 

( 10 ) 


where V is the set of K*s for which the limit in (10) exists and is finite. 

Note that equivalent for optimization; i.e., the 

optimal control gain for both cost functions is the same. However, as shown 
in Lemma 2, the limit in (10) exists and is finite when the closed-loop system 
is stable. 

In the remainder of this report, it will be assumed that Q and R are non- 
negative definite, unless specified otherwise. Lemma 1 is given without proof 
for completeness. 


Lemma 1 

Consider the system defined by (1) - (6). Then, 

= % tr I (Pj^^i(K) -Q) + Jq P.(K) w} 

N ^ ) 

+ h tv \ K' (r* P^(K) r + R)KV}, N^O (11) 

where 


Pi+l(K) = cf)(K)’ P^(K) (J)(K) + C K' R K C + Q, P^(K) = Q, (12) 
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([)(K) = (J) - r K C. 


(13) 


Furthermore, P^(K) f P(K) < «? if p(cf)(K)) < 1? 

The necessary conditions for the finite optimization interval, can be 
obtained from (11) and (12) by usual methods [10]. For the infinite optimi- 
zation interval, it is necessary to obtain a suitable expression for J(K). 


Lemma 2 

If p((j)(K)) < 1, J(K) is finite and is given by 

J(K) = h tr \ P(K) W } + tr I K' (F' P(K) T + R) K V ( , (14) 

P(K) = cj)(K)' P(K) (j)(K) + C' K' R K C + Q. (15) 


Proof: From (9), (10), and (11), 

J(K) = ^ lim tr 


N-^ 


S I 


N+1 o N+1 i=0 i 


P . (K) W 


+ tr| K’ (r Jq P.(K) F + R) K V } 


(16) 


By Lemma 1, P^^^j^C^) converges (to a finite matrix); hence, it is bounded, 
so that the term depending on vanishes as N-^. Since P^(K) is bounded and 
convergent, it can be shown that 


1 N 

lim ^ P.(K) = P(K). 

N-w N+1 1=0 1 


(17) 


Substituting (17) into (16) results in the desired expression. 


Thus, the cost J (K) (and P(K)) is finite on the set S of stabilizing 
feedback gains: 

S = I k|p(({)(K)) < l}. (18) 


p(<l)(K)) denotes the spectral radius of (f)(K). 
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If the system ((}), F, C) is output stabilizable, then S (hence V) is not empty 
so that the optimization problem is well-defined, and will be posed as: Find 

a K* in S such that 

J(K*) ^ J(K), K € S (19) 

It should be noted that this formulation, (19), of the optimization problem 
guarantees that a solution, when it exists, stabilizes the closed-loop 
system, by restricting the minimization to S, While most problems can be 
treated with this formulation, some cases of practical significance [24] 
require that the minimization be performed over V, This class of problems 
will be treated in future research. 

Although the cost function selected here does not contain the cross- 
coupling term between state and control which arises in sampled-data problems 
it is well-known that (e.g., [25]) a simple linear transfomation reduces 
this case to the one considered in this report. Thus, the results obtained 
here apply equally to the sampled-data formulation. 


111. EXISTENCE OF A SOLUTION 

This section considers sufficient conditions for the existence of a 
solution to the optimal control problem posed; i.e., a global minimum in 
An effort is made to obtain conditions which are simple to verify and cover 
a large class of systems. The existence conditions obtained are given in 
Theorems 2 and 3. Since the stability set S, over which the minimization is 
to take place, is an open and sometimes unbounded set, output stabilizability 
alone does not guarantee the existence of a stable solution. To guarantee a 
solution, it is necessary to determine conditions under which the optimal 
gain is an interior point of S. This is achieved in the following, by 



determining conditions which guarantee that the optimal cost is attained by 
a gain belonging to a closed and bounded subset of 5. The required conditions 
are given in Lemmas 7 and 8. First, it is of interest to show the continuity 
of the cost J(K). 

Lemma 3 

J(K) and P(K) are continuous on S. 

Proof; From (14), note that J(K) is continuous if P(K) is continuous. Thus, 
let K, K + AK € S; manipulating (15), 

AP(K, AK) = P(K + AK) - P(K) (20) 

= (})(K)' AP(K, AK) (()(K) 

+ C AK’ (R K C - r' P(K + AK) <()(K)) 

+ (R K C - r* P(K + AK) <|)(K))' AK C 

+ C’ AK' (r* P(K + AK) r + R) AK C (21) 

Now, let 3e = 1 - p(4)(K)) > 0, and select a matrix norm, say || • || , such 

that 


<|)(K)|| < p(<J)(K)) + e < 1. 


( 22 ) 


Using (22) and (21) , it can be shown that 

II AP(K, AK) II II C' AK' (R K C - r' P(K + AK) <f)(K)) 

1-|U(K)|| L 

+ (R K C - r' P(K + AK) (|)(K)) AK C 

+ C AK' (r' P(K + AK) r + R) AK C j|l (23) 
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As can be seen from (23), if P(K + AK) Is bounded in some neighborhood 
of K, then P is continuous at K. Using (22) and (15) 

11 P(K + AK) 11 < 11 C (K + AK)' R(K + AK) C + qH , 

1-11 4)(K + AK)lr 

11 (()(K + AK) 11 < 1 (24) 

Since the norm is a continuous function of the matrix elements, the set 
|k + AK € S II (|)(K + AK) II < 1 - e, and || Ak|[ < d}, for some positive 6, 
is a closed neighborhood of K over which P(K + AK) is bounded. Thus, from (23) 

lim II AP(K, AK)|| =0, K € S (25) 

AK->0 

so that P(K) and J (K) are continuous at K. 

Since S is an open set, and is not necessarily bounded, it is necessary 
to determine conditions under which the infimum of J (K) is attained at some 
^ S; i.e., sufficient conditions for the existence of an optimal solution. 
First note the following significant property of output stabilizable systems, 
whose proof is given in the Appendix. 

Theorem 1 

Let (C, ({), r) be output stabilizable. Then C (J) F 0 if, and only if, 

p((|)) < 1. 

Loosely, this property of output stabilizable systems states that the 
unstable modes of the system must be simultaneously observed in the output 
(which is used for feedback), and excited (or reachable) by the control which 
will stabilize the system. 

Definition 

j^(K) = h tv { P^(K) w } + tr I K' (r’ Pjj(K) r + R) K V [ , (26) 
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(27) 


- SI lio '■i*) 


The difference between 3^00 and the cost Jjg(K) is the term depending 

on the initial covariance S , as can be seen from 

o 

^ Y(N+1) ^ ^ 

However, it is preferrable to work with to its monotonicity, as 

shown in the following lemma. 


Lemma 4 

For any gain K, j^(K) is non-decreasing; i.e., 

0 ^ ^ Jn+i(K) . (29) 

Proof: By Lemma 1, P^j(K) is non-decreasing. It follows that Pj^(^) is also 

non-decreasing, since 

^ ^ ‘’n®> «»> 

■ SI ^N-1** SI ’’n® ^ SI 

It follows that and K’ (F’ F + R) K are both non-decreasing. 

Using Lemma A5 in the Appendix, we obtain the desired result given by (29) . 

Lemma 5 

Let K, € 5. Then for each G > 0 and integer N, there is an 

integer i^ such that 

3M < j(K ) + G . (32) 
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Proof: Note that is a continuous function of K over R™. Hence, 

JnCKi) ^ Thus, given N and e > 0, there is an integer, say 

i^,, such that whenever i i.^ 

N N 

VhJ - " " ^ 3N»i > + " ^ 

N N N 

Corollary 1 

Let K. € S, K. ^ K. Then 
1 1 


1) if j (K^) is bounded by B, then j ^ B (35) 

2) if j (K^) j*, then jjj(K) i j (K) ^ j* (36) 

3) if jjj(K) f *”> then j(K^) is not bounded . (37) 

Proof: 1) let j (K^) ^ B, then for e > 0 

j (K) j(K. ) + e < B + e (38) 

N 

/. j (K) B + e, ve > 0 (39) 

(35) follows, by letting e 0. 


2) Let j(K.) j*. Then, for any subsequence | K. , N i 0 |, j (K. ) -*■ j*; 

hence as N oo (32) results in 

j (K) < j* + e. VG > 0 (40) 

As e + 0, (36) follows. 

3) If j^(^) ^9 then j(K. ) f so that j(K.) cannot be bounded. 



Lemma 6 


Let (C, (j), r) be output stabilizable, Q ^ eC’C, W 2 eFF’ for some e > 0. 
Then J^(K) ^ t K / S. 

Proof; Recall that 

Pj^(K) = /(K)' Q (|)^(K) + (j)^(K)' (Q + C K' R K C) (|)^(K) (41) 

Now note that 

tr I Pjj(K) W [ ^ tr I Jg 4>^(K) * Q (|)^(K) W [ 

^ tr { 4>^(K)' C C (|)^(K) W [ (42) 

e tr| C (f)^(K) W <|)^(K)’ C'} 

> tr| C <})^(K) rr* 4>^(K)' C’} (43) 

th 

By Theorem 1, it follows that whenever K S, the N term of the series in 
(43), i.e. , tr I C (j) (K) FF’ (p (K) ’ C’ [ does not tend to zero; hence the series 
increases without bound. It follows that 


tr { P^(K) W } 

•f* OO 


(44) 

j^(K) > h tr 

{Pj^(K) W} 

' 2W+1) llo 

(45) 


Since is monotonic, using (28) the desired result follows. 

Lemma 6 can be interpreted as stating that, for an output stabilizable 
system, if each output variable is penalized and each control variable is 
corrupted by noise, then the cost is infinite unless the closed-loop system is 
stable. Thus, the optimal gain, if one exists, has to stabilize the system. 
Conversely, if each output variable is penalized and each control variable is 
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corrupted by noise, the stability of the closed-loop system is determined com- 
pletely by whether the cost is finite or not. Note that, in this case, V = S. 

Lemma 7 

Let (C, (}), r) be output stabilizable and for some e > 0, let Q ^ eC’C, 

W ^ err'. Then S(a) = { K « S|j(K) ^ a}, a « R is closed. 

Proof: Let € S(a), and ^ K. If K < 5, then (by Lemma 6) j^(K) f 
so that by part (3) of Lemma 5, J is not bounded, a contradiction. 

Hence, K c 5. Since J is continuous on S, a > J(K^) J (K) ^ a; K € 5(a) 

Lemma 8 

If (r' Q r + R) > 0 and (C W C + V) > 0, then S(a) = |k € 5|j(K) ^ a } 
is bounded, a € R. 


Proof: Suppose for some a e R, 5(a) is unbounded. Then it contains an unbounded 

increasing sequence, say C 5(a). Now note that, using identities on the 

trace, the cost J(K) can be rewritten as 


J(K) = Js tr I Q S(K) } + % tr { R K(C S(K) C' + V) K' K « S (46 a) 


S(K) = (J)(K) S(K) (f)(K)' + r K V K' r' + W, K « 5 


(46 b) 


Now let P(K) = r* P(K) r + R, S(K) = C S (K) C' + V, and note that 


2J (K^) = tr I P (K^) W [ + tr I P (K^) V } 2a 


(47 a) 


2J(K^) = tr { 0 S(K^) [ + tr I R S(K^) [ < 2a 


(47 b) 


Note that P(K) i r' Q r + R, S(K) ^ C W C + V, P(K) ^ (})(K)' 0 (})(K). Thus, 


tr { K^(r' Q r + R) v} ^ 2a, 


(48) 
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( 49 ) 


tr I R (C W C + V) ^ 2a, 
tr I c()(K^)' Q (})(Kp W} ^ 2a. 


tr 


Since 


I 


K, 

X 


K. 


k! k. 

— (F' Q r + R) — V 


K. 


s: 2a/ II i 0; 


(50) 


(51) 


is bounded it has a limit point, say K, such that K =1 and 


tr I K' (F' Q F + R) K V } = 0. 


(52) 


Similarly, 


tr I R K(C W C + V) K' J = 0, 
tr ] K' F' 0 F K C W C' } = 0, 


(53) 

(54) 


where (54) follows from (50) by noting that -F K C is a limit point of 
](J)(K^)/|| K^ll Thus, unless some K ^ 0 satisfies (52), (53) and (54), S (a) 
is bounded. Now (52) - (54) can be manipulated to obtain 

tr I K' (F' Q F + R) K(C W C' + V) [ = 0. (55) 

The desired result follows by noting that when (F* Q F + R) and (C W C' + V) 
are positive definite, the unique solution of (55) is K = 0; so that S(a) 
cannot be unbounded. 

Theorem 2 

Let (C, (p, F) be output stabilizable, (F* Q F + R) and (C W C’ + V) be 
positive definite, and for some e > 0, Q ^ e:C*C, W ^ eFF*. Then there exists 
a K c S such that 

J(K*) < J(K), K e S. 
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Proof: Let J* = inf J(K) < Necessarily, there is a sequence K., i i 0 

KeS ^ 

in S such that J(K^) 4- J*. By Lemma 8, =| K ( S|j(K) < J(K^) }■ is bounded. 

By the Bolzano-Weierstrass theorem, | | has a limit point, say K*. By Lemma 

* 

7, S is closed, so that K e S. 
o 

An optimal solution which necessarily stabilizes the closed-loop system 
is seen to exist for the large class of problems which meet the conditions 
required by Theorem 2. It is of interest to note that these conditions in- 
elude the cases of no measurement noise (V = 0) , and no control penalty (R = 0) 

A. 

as well as V = R = 0 simultaneously. Furthermore, no restrictions are placed 
on the relative magnitude of the number of states, n, the number of measure- 
ments, m, and the number of controls, r; the ranks of F and C are also 
arbitrary. Thus, multiple measurements of the same variable (i.e. , C does 
not have full rank) or cases where there are more controls than measurements 
(or states) will have an optimal stable solution if the existence conditions 
are met. Then, the optimal gain may be obtained using the algorithm presented. 

Theorem 3 

A 

Let Q and W be positive definite, F and C have full rank, and m ^ n, 
r < n. Then J (K) has a finite minimum over V if, and only if, ((|), F, C) is 
output stabilizable , The minimal gain stabilizes the system. 

Proof: Suppose (({), F, C) is not output stabilizable, K c R^^. Then let^ 

X 6 be a normalized eigenvector of (|>(K) corresponding to an eigenvalue X, 
such that |X| > 1. From Lemma 1, Eq. (12), note that 

X® X ^ |a1^ P^(K) X + Q X ^ (56) 

“3 

C is the set of n-dimensional vectors with complex components. 


4 


H denotes the complex conjugate transpose. 



a| > 1 


(57) 


X® P^(K) X > (i + 1) X® Q X > 0, 


where (57) is obtained by solving (56) and the fact that Q > 0. It follows 
that 


p(P^(K)) ^ x^ P^(K) X ^ (i + 1) x^ Q X ^ (i + 1) m(Q), 


(58) 


where m(Q) > 0 is the smallest eigenvalue of Q; so that using (11), 

V^'> ^ ife Jo “1 ‘’i<« “t ^ Jo + “• 


(59) 


Thus, J(K) is not finite for any K t R™, which shows necessity. For 
sufficiency, first note that we have also shown that S = Vy since whenever 
p(^(K)) ^ 1 the limit in (10) exists but is not finite; i.e., PCS. Thus, 
if ((p 9 r> C) is output stabilizable 


J* = inf J(K) = inf J(K) < oo. 
K(V KeS 


(60) 


Note that F' Q F > 0, and C W C' >0. Now since Q > 0, if 0 < ^ m(Q)/p(C'C) 


x' C'C X ^ p(C'C) x'x ^ m(Q) x'x ^ x' Qx, x « r’^ . (61) 

Similarly, if 0 < e 2 ^ ni(W) /p(rr* ) , then 


rr' ^ m(W) I ^ W. 


(62) 


Letting e = min (e^^, ^2^ * seen that all the conditions of Theorem 

2 are met, so that a minimum in S exists. 


Theorem 2 and Theorem 3 show that measurement noise or control penalty 
terms are not necessary for a solution to exist, which is a major difference 
between the discrete and continuous output feedback problems. For the 
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continuous deterministic as well as stochastic problems, a solution does not 
exist when the control penalty term vanishes; i.e., R = 0, as can be easily 
verified by considering a first order example. Furthermore, note that output 
stabilizability is not a necessary condition for the existence of a solution 
in discrete or continuous problems when Q is not positive definite. This can 
be verified by the trivial counterexample of an unstabilizable system with 
Q = 0 and R > 0, which has a minimum at K = 0, J(0) = 0 < Non- trivial 

counterexamples, where Q ^ 0 but singular, can be easily constructed for 
systems of order greater than 1. Finally, while the existence conditions 
given are not necessary and can be further generalized, they cover a broad 
class of systems and are simple for verification purposes. 

Whereas the question of existence has been satisfactorily treated, and 
the development of a reliable algorithm will be treated in Section V, "signif- 
icant" results on the uniqueness of the optimal solution are not available, 
and require further attention. 

The existence conditions obtained are summarized below for ease of 
reference. 

E-1. (C, (J), F) is output stabilizable, (F’ Q F + R) > 0, (C W C’ + V) > 0, 

Q > eC’C and W > eFF’,e > 0. 

E-2. (C, (J), F) is output stabilizable, Q > 0, W > 0, F and C have full rank, 
m ^ n, r < n 

If one of the above conditions holds, a stable global minimum exists. 

It should be noted that the class of problems covered by E— 2 is a subset of 
the class covered by E-1. 



IV* INCREMENTAL COST AND NECESSARY CONDITIONS 


The necessary conditions for the optimal discrete/continuous, stochastic/ 
deterministic output feedback problems have been previously explored [l] - 
[13] * These conditions have usually been obtained using the Langrangian 
approach which requires the differentiability of the cost function. As the 
resulting equations are coupled nonlinear matrix equations, a reliable 
method of obtaining their solution has not been available despite numerous 
efforts . 

The approach taken here is not to solve the necessary conditions, but 
to obtain a gain which minimizes the cost function. The necessary conditions 
are not required (directly) for the development of the algorithm, but are a 
by-product of the development. The optimal gain, however, is a solution of 
the necessary conditions. Thus, the approach taken is to obtain an expres- 
sion for the incremental cost; i.e., the change in the cost function due to a 
change in the gain. The following important lemma provides the desired 
expression. 

Lemma 9 

Let K and K + AK be in S; then the incremental cost is given by 

AJ(K, AK) = J(K + AK) - J(K) (63) 

= k tr 

+ AK’ P(K + AK) AK S(K) j. (64) 

where 


■|2AK’ j^P(K + AK) K S(K) - F’ P(K + AK) <f> S(K) C'J 


18 



(65) 


P(K) = r' P(K) r+ R, S(K) = C S(K) C + V. 

Proof; Let K and K + AK be in S. Substituting (14) into (63), we obtain 

AJ(K, AK) = h tr [ AP(K, AK) W } + % tr { 2AK' P(K + AK) K V 

+ AK' r' AP(K, AK) r K V + AK' P(K + AK) AK V | (66) 

Substituting the infinite series solution of (21) into (66) and using 
trace identities, it can be shown that 

AJ(K, AK) = h tr |2AK' ^P(K + AK) K V + (R K C - T' P(K + AK) (f)(K)) S(K) C' 
+ AK' P(K + AK) AK S(K) } (67) 

Rearranging the linear terms results in (64). 

The usefulness of Lemma 9 is largely due to its generality. Note that 
the incremental cost function AJ(K, AK) is the total change in J (K) , not the 
first order variation. Also note that the only restriction placed is that K 
and K + AK belong to the stability set S. This condition ensures that ambig- 
uous terms of the form oo - oo do not appear in the proof. This generality 
makes Lemma 9 useful in dynamic compensation and decentralized control problems 
as well as the output feedback problem considered here. An immediate conse- 
quence is given in the following Lemma. 

Lemma 10 

J (K) is continuously differentiable on S, and 

|^(K) = P(K) K S(K) - r' P(K) S(K) C , 


K t S 


( 68 ) 



Proof: From Lemma 3, recall that P(K), hence P(K), is continuous on S. By 

observation of (64) and the definition of differentiation, we obtain (68) 
which, by Lemma 3, is continuous. 

It may be noted that the gradient itself is differentiable on S, and 
that, in fact, J(K) has derivatives of all orders on S. Since J(K) is con- 
tinuously differentiable on S, if a minimum in 5 exists, then the gradient 
must vanish; hence, 

(P’ P(K*) r + R) K*(C S(K*) C’ + V) = T' P(k') c() S(K^ C', K* € S, (69) 

where P(K ) and S(K ) satisfy (15) and (46 b) , resp. Thus, (69), (46 b) and 

(15) are the necessary conditions. It is clear that when E-1 or E-2 hold, the 

* 

necessary conditions have at least one solution, K € 5. 


V. DEVELOPMENT OF THE ALGORITHM 

While the necessary conditions for the various formulations of the output 
feedback problem (e.g., continuous and stochastic, discrete and deterministic, 
etc.) are well-known, the unavailability of a reliable algorithm to determine 
its solutions has been the major hindrance to the successful application of 
optimal output feedback and dynamic compensation to design problems of prac- 
tical significance. The algorithm developed in this section is shown to 
provide a systematic method of obtaining a solution to the necessary conditions 
for the class of problems which satisfy one of the existence conditions. 
Furthermore, the authors’ experience with non-trivial systems indicates a 
**fast" rate of convergence, as is discussed in the next section. The following 
theorem provides the basis for the algorithm. 
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Theorem 4 (Convergence) 


Let one of the existence conditions E-1 or E-2 hold, and let K be in 5. 

o 

Then there exist 3 * (0, 1] and K in S such that 

J(K^) 4- J(K*), and |^(K^)-> ||(K*) = 0, (70) 

whenever 0 < a < 3 and the sequence i ^ 0 } is defined by 

K.^- = K. + a d(K.), i ^ 0 (71) 

1+1 1 1 

d(K.) = P(K,)"^ r* P(K.) (}) S(K.) C* S(K.)"^ - K, , K. t S (72) 

1 1 1^1 1 1 1 


Proof: Consider the inverse image 

[^0, J(K^)] ^ = |k 6 S|0 ^ J(K) ^ J(K^) I (73) 

By Lemma 7 (also note (61) and (62)), is closed. Recall that if E-1 or 
E-2 holds, then is also bounded, by Lemma 8. Now, to show that for some 
a > 0, the set 

= |k + a d(K) € R™1 k € a e [0, a] | (74) 

is a subset of 5, suppose that no such a > 0 exists. Then it is possible 
to construct a sequence a^ + 0 and a sequence] that 


p((})(K^ + a^ d(K^))) > 1. 


(75) 


Since is closed and bounded, by the Bolzano-Weierstrass theorem | j- 
has a limit point, say K in 5^. Now note that if E-1 or E-2 holds, then 


P(K) 

i r’ Q r + R > 0 , 

K e S 

(76) 

S(K) 

^ c w C + V > 0, 

K e 5 

(77) 
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It follows that P(K) ^ and S(K) and hence d(K), exist and are continuous 

on S; so that d(K) is continuous and bounded on the closed and bounded set S . 

o 

Since p((j)(K)) is also continuous, for some subsequence 
K. + a. d(K, ) K, 

1. 1. X. 

J J J 

p(c|)(K^ ) + d(K^ )) p((()(K)) > 1, (78) 

j j j 

which is a contradiction since K belongs to S . It follows that S C S 

o oa 

for some a > 0, which will now be considered fixed. 

From its construction (see (74)) and the continuity of d(K), it can be 
shown that is closed and bounded. Since P(K) is continuously differen- 
tiable over the closed and bounded set S , it can be shown that for some 

oa 

< oo 

II AP(K, a d(K))|| < a M II d(K)|| , K € a € [0, a]. (79) 

Rearranging the expression for the cost increments given in Lemma 9, we 
obtain 

AJ(K, AK) = h 2AK' -^(K) + AK' P(K) AK S(K) 

+ 2AK* r' AP(K, AK) |^r(K + AK) S(K) - (|) S(K) C' 

K, K + AK « 5 (80) 

Using Lemma 10 in (72) 

d(K) = - P(K)~^ ||(K) S(K)"^, K « (81) 

Now set AK = a d(K) in (80). As shown above, K + a d(K) belongs to S C 5 

oa 

whenever K is in so that (80) can be rearranged in the form 
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AJ(K, a d(K)) = Is [- a(2 - a) A(K) + B(K, a)], 


K e S , a < (0, a] , 

o 


( 82 ) 


where 


A(K) = tr| S(K)"^ |^(K)' P(K)"^ ||(K) | 

= tr|d(K)' P(K) d(K) S(K)|^ 0 
B(K, a) = I tr|d(K)’ r' AP(K, a d(K)) 

[r(K + a d(K)) S(K) - (j) S(K) C ] } 


(83) 

(84) 


(85) 


Using (76), (77), (81) and (84) it can be shown that for some M 2 > 0 


A(K) > M„ ||d(K) If ^ 0, K f S 


( 86 ) 


On the other hand, using (79) and (85) 


;(K, a) I ^ l|d(K)|r, K f S^, 0 ^ a ^ a 


(87) 


for some It follows that 


M, 


1b(K, a)| < ^ A(K) = A(K), K < S^, 0 < a < a. (88) 

Select 3 in (0, 1] such that 3 ^ a and 3 ^ ^ satisfy 

0 < a ^ 3 . Now substitute (88) into (82) 


AJ(K, a d(K)) 




a A(K) + a A(K) 


= % - a(l - a M^) A(K)J , K f S^, 0 < a < 3 


(89) 


3 T 

Since A(K) > 0 whenever "^(N^) ^ 0> 0 < a < 


AJ(K, a d(K)) < 0, K e S^, 0 < a < 3 


(90) 
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f (K) = K + a d(K) , 0 < a ^ 3 (91) 

i.e,, if K e then f (K) e 5^. It follows that the sequence [ defined by 
(71) is a subset of 5^, and J(K^) is monotonic and bounded, and necessarily 

“A 

converges, while | has a limit point, K ,in S^; hence, 

0 ^ J(K^) + J(K*). (92) 


The increments AJ(K^, a d(K^)) must then converge to zero. Combining ( 86 ) 
and (89), 


0 < ||d(K.)|p < A(K.) < ct(i _^a M 3 ) ° 


SO that A(K^) and d(K^) vanish. From (81) ^(^^) Oj since any subsequance 

9 

also converges to zero and the gradient is continuous (Lemma 10) "^(^^ ) = 0, 
which completes the proof. 


It is seen that for a broad class of problems, it is possible to construct 
a sequence of gains whose costs monotonically decrease, while the corresponding 
gradients converge to zero, if any stabilizing gain is available. It is 
guaranteed that the sequence of gains has a limit point, K , which stabilizes 
the closed loop system and satifies the necessary condition; i.e., K is a 
critical point of J(K). Thus, it is possible to find a stabilizing gain whose 
gradient is as small as desired. Furthermore, any stabilizing gain can be 
used to start the algorithm. An initialization procedure is discussed in [26] . 

An important aspect of the method is due to the invariance of the set 
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under f(K) given in (91). This property insures that the new (successor) 
gain will not fall outside the stability region. It is also significant that 
a constant value of the parameter a (appropriately selected) is sufficient to 
obtain the convergence properties needed. This property makes it unnecessary 
to conduct lengthy line searches at every iteration. Finally, it is the 
exploitation of the specific form of the cost function, and, in particular, 
the "almost quadratic" form of the incremental cost that suggests the use 
of the direction d(K). This special form of the cost increments makes it 
unnecessary to compute or approximate large order (rm x rm) Hessian matrices 
which are often used in gradient search methods. 

Theorem 4 suggests the following algorithm. The objective is to choose 
a large positive a ^ 1 in (91) which makes the algorithm stable. The initial 
a chosen by the designer may be too large. Steps 4 and 7 check for lack of 
convergence and Step 8 decreases a using z. Theorem 4 guarantees that only a 
finite number of decreases in a will be required to arrive at a value of a for 
which the algorithm is stable. The equations in Step 2 and Step 3 can be solved 
using the Bartels-Stewart algorithm available in control software packages 
such as ORACLS, [27]. 

Algorithm 

Step 1 Choose K so that <b(K )= 6 - T K Cis stable, a € (0, 1], 

o o o o 

z > 1, and set i = 0. 

Step 2 Solve the following equation for S(K^) 

s(Kp = <{)(K^) s(K^) (j)(K^)* + r V r’ + W 

Step 3 Solve the following equation for P(K^) 

P(K^) = 4>(K^)’ P(K^) <J)(K^) + C’ R C + Q 
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Step 4 


Step 5 


Step 6 


Step 7 


Step 8 


Compute P(K^), S(K^) 


p(K^) = r' p(K^) r + R 


S(K^) =C S(K^) C + V 


Invert P(K^) and S(K^) using Cholesky decomposition. If either of 

these symmetric matrices is not positive definite, go to Step 8. 

Compute K , d(K.) 

new 1 

\ew " 4> S(K^ C S(K^)"^ 


d(K.) = K - K. 
1 new X 


Compute K 


1+1 


K. = K. + a. d(K.) 
1+1 1 11 


Evaluate Cost function 


J(K^) = h tr jP(K^) W } + % tr I K!^ (F* P(K^) (|) + R) V | 

If i = 0 set i to 1, = oc^, and go to Step 2 

Check to see if the algorithm is stable for the selected a: 

If J(K^) is negative go to Step 8 

If any element along the diagonal of S(K^) or P(K.) is negative 
go to Step 8 

If J(K^ - is negative go to Step 9, otherwise go to Step 8 

Decrease a 


a. = a. /z 

1 i' 

Go back to previous stabilizing gain 
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Ki . K^_^, d(K.) . d(K,_^) 


Compute K, . , : K . = K. + a. d(K,) 

^ 1+1 1+1 111 


Set ” *^i’ ^ ~ Step 2 


9J, 


Step 9 Compute 


9J 


-|^(K^) = P(K^) S(K^) - r' P(K^) (P S(Kp C 


IF 




and J(K^) - 


are less than some convergence criterion 


STOP otherwise set i = i+1 and go to Step 2 


VI. APPLICATION TO AN AIRCRAFT OUTER-LOOP DIGITAL CONTROL DESIGN PROBLEM 

The output feedback control system design methodology in the previous 
section is currently being used to design an outer-loop control system for a 
typical small transport jet aircraft. The purpose of the outer-loop system 
is to feedback guidance errors to the inner-loop control system so that the 
aircraft tracks a 3-dimensional flight path. Initial results from the outer- 
loop synthesis are presented to illustrate properties of the output feedback 
algorithm and to compare the algorithm to other numerical approaches. 

The example given here is the design of the horizontal path following 
outer-loop feedback gains for a proposed set of lateral-direction guidance 
error signals. The lateral digital inner-loop control system has previously 
been designed and extensively flight tested. The lateral, inner-loop control 
system feeds back filtered roll rate, p, and roll angle, (f), to the aileron/ 
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spoiler command, 6*^, and feeds back washed-out filtered yaw rate, r , to 

the rudder command, 6 . A block diagram of the continuous time inner-loop 

KC 

control system before the Tustin transformation is used to obtain the digital 
implementation is shown in Figure 1. 

The continuous- time linear model of the aircraft with the closed inner- 
loop system has the eigenvalues shown in Table 1. The aircraft model is 
determined with the aircraft trimmed on a three degree glideslope at 64 m/s. 
The model includes six aircraft states (Av, Ar, Ap, A(|), Aipy Ay), two gust 

/N /\ /s 

states (Awj^, Aw^) , four inner-loop filter states (Ap, A({)^, Ar, Ar^^) , and 

the aileron actuator state (A6 ) for a total of thirteen states. The controls 

a 

are roll angle command, A(|)^, and rudder command A6^^. The gust terms are 
modelled using the well-known Dryden spectrum. The white noise in the gust 
model and in the model of the inner-loop measurements are used to determine 
some of the elements in the continuous-time process noise covariance matrix, 

W. All of the elements along the W diagonal are shown in Table 2. The 
discrete time process noise covariance matrix W is determined from the matrix, 
W, and the plant dynamics represented at the sampling instants [25]. 

A set of horizontal path guidance errors signals being investigated uses 
yaw angle error, Atf^, lateral position error Ay, and lateral velocity error 
for feedback. These error signals determine the elements in the observation 
matrix, C, the quadratic weighting elements in Q, and the observation noise 
matrix, V. Values for the continuous time diagonal weighting elements in 
Q and R are shown in Table 2. After these elements are specified, the con- 
tinuous-time problem is transformed to an equivalent discrete-time problem 
using the sampled-data regulator [25] . 

Three algorithms are used for comparison purposes to solve the output 
feedback problem just presented. Algorithm I is the discrete version of the 
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numerical approach presented in [5] and discussed as Algorithm .2 in [20] and 
[21]. Algorithm I is explicitly discussed in [13]. Algorithm I is actually 
a special case of the algorithm in Section V where the latter is constrained 
to use a = 1.0. The continuous time version of the algorithm is known to be 
divergent for specific examples [20]. Algorithm II is the discrete version 
of the Davidon Fletcher Powell algorithm (discussed in [14] , [19] , and called 
Algorithm 4 in [21]), where the Hessian is restarted every N steps with a 
positive definite priming matrix. Algorithm III is the one presented in 
Section V. The starting stabilizing feedback gain for all three algorithms 
is obtained using an output feedback pole placement procedure discussed in [26] . 

Table 3 compares the three algorithms for two starting values of the 

stabilizing output feedback gain, K^. Test 1 uses "far** from the optimal 

gain. In both tests Algorithm I diverges. Algorithm II converges slowly and 

was prematurely terminated in Test I because of the excessive computations 

needed to reach a minimum. Algorithm III was started far from the optimal 

gain with = 1.0. Stability of the algorithm was obtained when was 

eventually reduced to 0.75 in Step 7. The algorithm converged in 44 itera- 

-4 

tions in Test 1 to an optimal gain using a convergence criterion of 10 
Algorithm III performed considerably better than Algorithms I and II when 
is far from the optimal gain in Test 1. Algorithm II is known to have better 
performance if the starting point is close to the minimum value. As shown in 
Test 2, however. Algorithm III still performs better than the gradient based 
approach of Algorithm II for chosen close to the optimum gain. Algorithm 
III reduced a to 0.75 and reached a minimum using 12 iterations requiring 
approximately a quarter of the amount of computation time needed by Algorithm 
II to reach a minimum. 
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The effect of varying the noise covariance matrices is shown in Table 1. 

If only the measurement noise is changed the effect is to decrease the feed- 
back gains on noisy measurements while increasing the feedback gains on measure- 
ments with reduced noise* If the large terms in W in Table 2 are removed, the 
result is shown in the last row in Table 1. The gains are reduced considerably. 
The best average long term stochastic performance for a plant with low process 
noise is to use little control activity. Non-zero initial conditions, however, 
may require a long time to asymptotically return to zero but ultimately con- 
tribute nothing to the infinite-time averaged stochastic cost function. The 
conflict between short term desirable transient response and long term stochas- 
tic performance will be addressed in greater detail in future efforts. 

VII. CONCLUSIONS 

The problem of designing an optimal instantaneous output feedback controller 
for a stochastic discrete-time system has been considered. Two sets of sufficient 
conditions for the existence of a minimum of the cost function are derived (E-1, 
E-2). The optimal controller is realized by a feedback gain matrix which is the 
simultaneous solution of three coupled matrix equations ((15), (46 b) , (69)). A 
computational algorithm to obtain an optimal gain is presented. If (81) is sub- 
stituted into (71) the algorithm can be interpreted as a cross between sequential 
methods and gradient search methods for determining the output feedback gain 
which minimizes the cost function. The algorithm produces a sequence of gains 
which monotonically decreases the cost function using a special direction obtained 
by a transformation of the cost gradient. A line search is avoided by showing 
that there is a fixed constant positive scalar in (71) which guarantees the 
sequence of gains has a limit point satisfying the necessary conditions for 
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optimality. A non-trivial 13th order aircraft digital control design example 
is used to show that the algorithm converges faster and more reliably than a 
sequential and gradient search method for determining the optimal output feed- 


back gain. 
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APPENDIX PROOF OF THEOREM 1 


This appendix contains a detailed proof of Theorem 1 which was used in 
obtaining existence conditions for the optimal output feedback problem. This 
property of output stabilizable systems states that the stability of the closed- 
loop system is completely determined by the convergence of C c()(K)^ T to zero. 
This property is used to show that S(a) is a closed set under the conditions 
given in Lemmas 6 or 7. The proof of Theorem 1 is relatively straightforward 
when the eigenvalues of cj)(K) are distinct. The general case of Theorem 1 
allows (1)(K) to have multiple eigenvalues. The general case of Theorem 1, which 
is necessary to obtain the results in the report requires a more detailed and 
lengthy derivation and is presented in the remainder of this appendix. The 
following Lemmas are used in the proof of Theorem 1. 

Lemma A1 

Let (C, (J), r) be output stabilizable. Then for any K € R™ 

(|)(K) X = Xx, and x^C^Cx=0->- |X| <1, and (Al) 

(j)^(K) z = Xz, and z^rr^z = 0“^ |X| <1. (A2) 

Proof: First note that for any K, (C, cJ>(K), F) is output stabilizable. 

Thus, it suffices to show (Al) and (A2) for K = 0. Now suppose 

(f)x = Xx, C X = 0, and |X| ^ 1. (A3) 

Then, 

(cj) - r K C)x = (|)x - F K C X = Xx, lX| > 1, VK. (A4) 

It follows that (C, (J), F) cannot be output stabilizable, a contradiction. 
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Similarly, if 


(J)^ 2 = Xz, (f)^ z = 0, and X ^ 1, 


then 


((|) - r K c)^ z = 4)^ z - K ^ z = z, |x| ^ i, vk. 


Thus (C, (J), T) is not output stabilizable, a contradiction. 


Lemma A2 


Let X be an eigenvalue of (j) such that 


(t)x. =Xx.+x..-, l^i^p, x,T=0. 

1 1+1 ^ p+1 


Then 


9 X. 


k=0 ^kn ^ ^i+k’ 


l<i^p, n^O, 


b, . 1 == t), + b, . , b, = 6 , b - =0, b. = 1; 

k n+1 kn k-1 n ko ko -In On 


b =0, k > n. 
kn 


Proof: First note that (A8) holds for n = 0 and n = 1. Now assume 

(A8) holds for an arbitrary n; then 

.n+1 . P“i , .n-k P"^ , .n-k /-x , \ 

<■ ■ 'f' kSo \n * >^l+k ■ kSo \n ’'l+k =‘i+k+l> 


P-i n+l-k p-i+1 

= ,I. b. X" ^ ^ b^, 


k=0 kn 


i+k ‘ k'=l “k'-l n ^ ^i+k' 


_ P^i , , n+l-k . P;i , 

" k=0 ^kn ^ ^ 


n+l-k' 

i+k ‘ k'=0 “k'-l n ^i+k' 




n+l-k 


= , (b, + b- ^ ) X X. , 

k=0 kn k-1 n i+k 


(A5) 

(A6) 

(A7) 

(A8) 

(A9) 

that 

(AlO) 

(All) 

(A12) 

(A13) 
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so that (A8) holds for n+1. 


Note that the bj^^'s are the binomial coefficients, is an eigenvector 
and x^is a principle vector when i = 1, 2, p. Now let J be the Jordan 

canonical form of (}). 


(p = X J Z, Z = X 


-1 


(A14) 


where the columns of X are the principal vectors of <(). Partition J, X, Z as 


Jl 


J = 


, X = (X^ x^ . . . X^) , z = 


(A15) 


>Zl 





1 < £ < L 


(A16) 


“£i " ^ ^£i’ ^£i = ^£i P=“J^P£> 


(A17) 


and define the unit step function H as 


H. 


ij 


1, i ^ j 

0, i > j 


Lemma A3 


c ({)" r 


n 


J p-1 

B., b, A. 
j=l k=0 jk kn J 


(A18) 
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J are the distinct eigenvalues 


where is given by (A9), X., j = 1, 2, 


of ((), and 


jk 






ih^ ^ik ’ 


X. ^ 0; 


X . =0; 
J 


(A19) 


P£-k 

^£k " p^-1 i=l i+k ^li ’ 




X^^, ^ I < (A20) 


Proof : 


C r = a Z (f)*^ X 3 = (j)"^ 6j 


~ ih “£ 


(A21) 

(A22) 


Since 


z„ 4)" X. = z„ X j" z X. = 0, 
J -f J 


Using Lemma A2, 


tn 


Noting that 


X. 


L 

= £Si 

F 

i=i 

F 

j=i 

A2, 



L 

= £Sl 

F 

i=i 

P£ 

L 

= £il 

F 

i=i 

k=0 

i+1 

• -J., . 

J i+k 

L 

" ih 

k 

k=0 

L 

Ji 

k=0 


j ^ . 




V' 


_n-k / p 


n-k 


"£i 


n-k 


H. 


p^-i i+1 ^£i ^kn 


n-k 


(A23) 

(A2A) 


(A25) 


(A26) 


(A27) 


(A28) 


(A29) 
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(A30) 


L Pc-“1 Vp ^ 

ih k=0 i=l “k p^-i i+k hi ^kn 


L Po-1 / Po-k \ n-k 

£=1 k=0 \ i=l h i+k hi) ^kn h 


L p -1 _n-k 

1=1 k=0 hk ^kn h 


(A31) 

(A32) 


where 


kk'"kp^-/iSN£l+l»£l' I” iSSl'’^ 


(A33) 


Now assume that the s have been ordered so that 


h ~ h^ ~ ^ ^ ^j+i ’ 


(A34) 


where 1=1^ < £- < ... < t , and the X.'s are distinct. Then (A32) can be 
1 ^ J J 

rewritten as 

C T = i, hi i A„, ^ XT^ b, x") , (A35) 

^ j=l k=0 y Ik I j kn J ’ 

which is the desired result. 

Lemma A4 


Let C (})’^ r 0. If [Xjl > 1, then 


B = 0, 0 < k < p-1 , 

3*^ 


(A36) 


where X., B., are defined in Lemma A3. 
3 3k 


Proof: First order the distinct eigenvalues X^ of (}> such that 


p((j)) = 1 X 1 > I X I > ... > I X. 


(A37) 


Suppose that 
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|Aj| ^ 1, j £ J^; |A^| < 1, j > 


(A38) 


Since is a polynomial in n of degree k. 


b, X . 0, J > i > » 0 < k < p-1 . 

kn j 1 


Hence, from the representation in Lemma A3, if C (J) T Oy then 

ij- B., b. x" 0 . 

j=l k=0 jk kn J 


Now consider 


•» ( liji)' . 


b, -> 0 
kn 


I Xj I < I ! , J > 


.^1 B., pV 
J=1 jk 


B,, p. 
3=1 jk J 


Hn-""' Pj , IS J SJ^i, 


“^0, 0 ^ k ^ p-1 


p-1 n 


Since 


p~l n 


k p-1 


0 < k < p-1 


-'ll 

,E, B. , p" 0. 

3=1 3 P-1 3 


Using (A46) in (A43) 
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(A47) 




'11 


, B., pV 

k=0 \j=l jk ^2 


b, -> 0. 
kn 


Repeating the same procedure, it follows that 


Jll 

B., p’? -> 0, 0 < k ^ p-1 . 

J=1 3^ J 


(A48) 


Now, note that the equations 


jSl ®jk ^ 0 < i < -1, 0 < k < p-1. 


(A49) 


can be expressed in matrix notation as 


P p"" B ^ 0 , 


(A50) 


where p = diag (p^) , 


Til 


(A51) 


/ ir " 


^11 


and where B is the matrix formed by the appropriate elements of the matrices 

B.- . Further note that p is invertible since the p.’s are distinct, 
jk ^3 


p"" B -> 0 , p^ B^^ 0 , 


iPjl ^ ’ '■’^'^11 


(A52) 


Bj^ = 0, 0 ^ k ^ p-1 , 1 ^ j ^ . 


(A53) 


Now substitute (A53) into (A40) to obtain 


E ftl B., p, a" 0 . 

i=J +1 ^=0 -1^ T 

3 


(A54) 
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PyX 

k=0 


3-j" +1 i ‘kn ’ 0. Pj - Jll < J ^ ^12 <A55) 


11 


11 


Repeating the same procedure as many times as necessary, it follows that 


= 0, 0 ^ k ^ p-1 , 1 ^ j ^ , 


(A56) 


which is the desired result. 

Theorem 1 

Let (C, (f), T) be output stabilizable . Then C (|)^ F 0 if, and only if 

pW < 1 . 

Comment: Note that if (C, <(), F) is output stabilizable then (C, <))(K)) F) 

is output stabilizable where (})(K) is defined in (13). The Theorem is used for 
the (C, (})(K), F) system in Lemma 6 but is proved here using (C, <|), F) . 

Proof: If p((f)) < 1, then (j)^ ^ 0; thus C (})^ F ^0. Now suppose that p((f)) > 1 
and C (})^ F 0; let be an eigenvalue such that |A_,| > 1. By Lemma A4, 


-k 

^k= £?£. ^£k^ =0, O^k^p-1 . 

X ' 


(A57) 


B 




j “jk l=Z. iSl i+k ^£i ^k p^-1 0 ^ k < p 1 . (A58) 


Now let == + <1 + Ij q ^ 0; i.e., A^ corresponds to q + 1 Jordan forms. 

It follows that Aj corresponds to q + 1 linearly independent eigenvectors, namely 
+i F£ 0 ^ i ^ q |. It is important to note the implications of Lemma A1 

j j 

to the multiple eigenvector case. Thus, let |x^, 0^i^q[bea set of linearly 
independent set of eigenvectors corresponding to the eigenvalue A; i.e., 


cf)x. = Ax, , 0 < i < q 

11 ^ 


(A59) 
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Then for any non-zero sequence { 0 ^ i ^ q} of complex numbers, the vector 

q 


X = a. X. 

1=0 1 1 


(A60) 


is also an eigenvector of ^ corresponding to A, 
Hence, Lemma A1 implies that 


Cx = ,Z^ a, Cx. 7 ^ 0 
1=0 1 1 


(A61) 


whenever \X\ > 1, provided that (C, (f), F) is output stabilizable • In other words, 

the set of vectors | Cx^, 0 < i < q [ is linearly independent. For the problem 

at hand, it follows that the set ^ ^ t = £.+q + l[ is 

' ^ 1 1+1 J ^ 

linearly independent. Similarly, it can be shown that | Bp , t. ^ t < I 

* L 1 J+L * 

is linearly independent. 

Now let p^ = max | p^, < t < [ » ^nd 1 < m < p^. We will show that 

(A57) implies 


B^i = 0, l^i^p^-p^+m, l^m^Pj, 


(A62) 


using finite induction. First note (A62) holds for m = 1, by evaluating (A58) 


for k = p , - 1. 
1 


7 

1=1. 

2 


l^+q 

J ^ 


p£-p.+l 

a„ ^ H_ , n = 0 

1=1 L i+p .-1 >ti p.-l Po“l 

1 1 L 


ih. “£ VI - - 0 


Pj 


(A63) 


(A64) 


where the fact that p^ - Pj+1 ^ 1 been used to obtain (A64). Since the 


linearly independent, it follows that 


^p^ p^ ^l± 1 ^ i ^ P£ Pj+1- 


(A65) 


Now suppose that (A62) holds for an arbitrary value m < p^ . Evaluating 


A3 



(A58) for k = Pj - m - 1, we obtain 

•^j+q p^-Pj+mfl 

y y 

1=1, ' ^ 


1=1 Z i+p,-in-l Zi p.-m-l po -1 

J J 


= 0 


(A66) 


Since (A62) holds for m, the only terms which might not vanish correspond 
to 1< i =P£-Pj +m+l. Hence (A66) becomes 

(A67) 


£j+q 

£=£. -c Pp t Pp-p.+nri-l p.-m pp 
J Z 3 J 


Since the p ’ s are linearly independent . 


n- =0, 1. ^ I ^ 1. + q . 

I p^-Pj+mfl p.-m P£ J J 

3^1 =0, 1 ^ i = p^ - p + m + 1; 


(A68) 

(A69) 


so that (A62) must hold for m + 1, and by induction for 1 ^ m ^ p^ . 
However, if (A62) holds for m = p^ , then 


3 « = 0 , 1. < I < 1.., 

^ Pi 2 1+1 


(A70) 


which, by Lemma 1, implies that (C, <}), F) is not output stabilizable, a con- 
tradication. Thus, we must have p(4>) < 1 which concludes the proof. 

Finally, an inequality that is used throughout the paper is given in Lemma 
A5 for completeness. 


Lemma A5 


Let P 2 » W be non-negative definite (and Hermitian) matrices such that 


Pj^ > P 2 > 0. Then 


tr |P^ w(> tr |P 2 W( > 0 


(A71) 
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Proof: Since W 2 0, let 

W = 0) 03^ (A72) 

tr|P^ W } = tr to[ > tr P 2 wf ^ 0. 

Note that whereas W need not be non-negative definite, its trace is 
non-negative. 
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Roll Rate, 
P> 

(Deg/Sec) 


Roll Angle 
(Deg) 


Phic, 



Aileron 

Command 

‘^AC’ 

(Deg) 


FIGURE 1 LATERAL-DIRECTIONAL INNER-LOOP BLOCK DIAGRAMS 











FIGURE 1 LATERAL-DIRECTIONAL INNER-LOOP BLOCK DIAGRAMS (CONCLUDED) 
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TABLE 1 EFFECTS OF CHANGING NOISE COVARIANCE MATRICES 


Design 

Parameters 


Outer-loop 

Open 

Inner-loop 

Closed 


Observation 
Noise: 
a, = 0.1 
= 3.05 
= 3.05 

y 


Observation 
Noise: 
a, = 0.32 
= 2.16 
a? = 0.67 

y 


Gain Matrix 
Au’ = [A(J) A6 ] 
Ax' = Ay^'^Ay] 



Observation 
Noise as Above 
Using Gusts & 
Inner-loop 
Measurement 
Noise Only 


-0.98 -0.016 -0.061 |(-0.32±j0.26) 

1.31 0.017 0.067 




(-0.32±j0.27) -0.155 -0.4A6 

-0.95±jl.2 

VO 

00 

CM 

1 

-5.0 

(-6.2±j2.0) 

-22.4 

(-0.059±j.014) -0.243 -0.914 

-0.89±j0.92 

-3.95 

-5.0 

(-6.2±jl.7) 

-22.4 







, Velocity-m/sec , Gusts-m/sec, 

Position-m 






The equivalent continuous-time closed-loop eigenvalues are obtained by applying first the log of a matrix 
to the discrete closed-loop plant then finding the eigenvalues of the resulting matrix. 





















TABLE 2 DIAGONAL ELEMENTS OF DESIGNED MATRICES 



R - Control Weighting 


Q - State Weighting 


6 

rc rc 


q,,=^0.25, q^^=0.25, q = 0.00093, q. . = 0.023 

W yy yy 

W - Process Noise 

a = 3.05, a = 0.3, a. = 0.6, a = 0.3 
wg p <p r 

W+ - Additional Process Noise (Initial Condition State Variance) 

a “ 1.5, a. = 5.0, o. = 5.0, a = 30.5, 

V w y 

= 1.0, = 1.0, = 1.0, a = 1.0 

r p (f) r 

^ ^ wo 


Angles-deg, Angular Rates-deg/sec , Velocity-m/sec , Gusts-m/sec , Position-m 
TABLE 3 ALGORITHM COMPARISONS 


Minimization 

Technique 


Test 1 


Initial Value 

for J(K ) 
o 

No. of Lyapunov 
Type Equations 
Solved 

Final Value 
for J(K) 


Test 2: 


Initial Value 

for J(K ) 
o 

No. of Lyapunov 
Type Equations 
Solved 

Final Value 
for J(K) 


Algorithm I 
(Algorithm III with a 



Diverged 


Algorithm II 
(Davidon- 
Fletcher-Powell ) 


179.048 


Algorithm III 
(Section V) 


179.048 

(a=1.0 to start) 



176.740 

Algorithm Prematurely 
Terminated Because of 
Poor Convergence 


173.643 


173.605 

Converged-Final 
a = 0.75 


173.643 

(a=1.0 to start) 


Diverged 


173.606 

Converged 


173.606 

Converged-Final 
a = 0.75 
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